fix(store): enforce case-folded uniqueness for account identities - #757
fix(store): enforce case-folded uniqueness for account identities#757rodboev wants to merge 3 commits into
Conversation
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
The account_identities primary key compares raw bytes while the application matches email-shaped identifiers case-insensitively, so two processes could insert case-variant rows for one logical identity and split its signal set. Persist the comparison-canonical form of each address in a new address_key column (NormalizeIdentifierForCompare: lowercased for email-shaped identifiers, verbatim otherwise) and enforce one row per logical identity with a partial unique index on (source_id, address_key). Store writers look rows up by the key, so both backends match under the same Go-owned rule, and a concurrent case-variant insert fails on the index and merges on retry. Previous-release binaries keep writing through the column's '' default, which the index exempts. Every store open derives missing keys and merges any case-variant duplicates under the identity-mutation lock, keeping the earliest confirmed_at, unioning signal sets, and bumping the identity revisions when rows collapse. Non-email identifiers (Matrix MXIDs, phone numbers, handles) keep byte-exact case-sensitive matching, address remains the case-preserved display value, and read-side attribution, cache, and query behavior is unchanged. Closes kenn-io#311
The duplicate-collapse repair refreshes message attribution, which folds identity matches into is_from_me. Running it before the provenance migration on a pre-provenance archive let the backfill read those identity-derived values as source-native and bake them into source_is_from_me permanently. Move the repair after the provenance migration; the ordering regression test fails at the earlier call site.
…ce timeout The duplicate-collapse repair refreshes source-wide message attribution, whose cost scales with archive size; under the pool-wide 30-second PostgreSQL statement_timeout a large upgraded source would cancel the repair and fail every subsequent open. Run it through runMaintenance like the other archive-size-scaled migrations. Create the partial unique index once per archive behind a migration ledger entry, also inside the maintenance escape hatch, so a lock held by a concurrent identity writer cannot trip the ordinary timeout during open; IF NOT EXISTS covers a cancellation between the create and the ledger write.
115c094 to
a3f11e8
Compare
rodboev
changed the title
roborev: Combined Review (
|
1 participant
The
account_identitiesprimary key compares raw bytes while the application matches email-shaped identifiers case-insensitively, so two processes (serveplus a parallel CLI command, or two writers on PostgreSQL) can insert case-variant rows for one logical identity. The duplicate rows split the identity's signal set and makemergeSignalSetsemantics break down, as described in #311.This change persists the comparison-canonical form of each address and lets the schema enforce the invariant the application already assumed:
address_keycolumn holdsNormalizeIdentifierForCompare(address): lowercased for email-shaped identifiers, verbatim for case-sensitive ones (Matrix MXIDs, phone numbers, handles).addressremains the case-preserved display value. The normalization stays Go-owned; no SQL reimplements the email-shape heuristic.(source_id, address_key) WHERE address_key <> ''enforces one row per logical identity for every keyed writer on both backends. A concurrent case-variant insert now fails on the index and merges into the existing row through the writers' existing retry loop.AddAccountIdentity, batch confirmation, the legacy-config migration) look rows up by the key, so SQLite and PostgreSQL match under one rule instead of backendLOWER()variants.''default, which the index exempts. On every store open, rows with missing keys are derived and case-variant duplicates merged under the identity-mutation lock: the earliest-confirmed row survives with its casing, signal sets union, and the identity revisions bump only when rows actually collapse.Read-side behavior (attribution, activity, caches, dedup) is unchanged; those consumers keep reading
address.Closes #311